1. Field of the Invention
The present invention relates to an image signal processing apparatus, and, more particularly, to an image signal processing apparatus that produces color difference signals from an image signal containing a plurality of color components output from a solid state image sensing device.
2. Description of the Related Art
FIG. 1 is a schematic block diagram of a conventional image sensing apparatus 100 having a CCD image sensor 1.
The CCD image sensor 1 has a plurality of light-receiving pixels, a plurality of vertical transfer shift registers and a single horizontal shift register. The light-receiving pixels, which are arranged in a matrix form on the light-receiving surface of the image sensor 1, produce and store information charges corresponding to the image of a sensed object (hereinafter the "target object"). The vertical shift registers sequentially shift the information charges, stored in the light-receiving pixels, in the vertical direction column by column. The horizontal shift register receives the information charges, transferred from the vertical shift registers, and outputs, row by row, an image signal I0 having a voltage corresponding to the information charges.
An analog processor 2 performs a process, such as sampling and holding or level clamping, on the image signal I0 from the CCD 1 to generate an image signal I1 of a predetermined format. For example, the image signal I0 has reset levels and signal levels alternately repeated in synchronism with the output operation of the CCD 1. In the sample and hold process, only an image signal having a signal level is extracted. The image signal I0 has a black reference level set every time the horizontal scan period ends. In the level clamping process, the black reference level of the image signal I0 is clamped to a predetermined level every time the horizontal scan period ends.
An A/D converter 3 performs A/D conversion of the image signal I1 from the analog processor 2. Specifically, the A/D converter 2 quantizes the image signal I1 in synchronism with the operation of the analog processor 2 (i.e., the output operation of the CCD 1) to produce image data D having a digital value corresponding to the information charges in each light-receiving pixel of the CCD 1.
A digital processor 4 performs a process, such as color separation or matrix operation, on the image data D from the A/D converter 3, to generate luminance data Y and color difference data U and V. For example, a color filter having colors arranged in a predetermined manner may be attached to the light-receiving surface of the CCD 1. In the color separation, the image data D is separated according to the arranged colors, producing a plurality of color component data. In the matrix operation, primary color data corresponding to the three primary colors of light are produced from the individual color component data, and are then combined by a predetermined ratio, thereby generating the color difference data U and V.
In accordance with a reference clock having a given period, a timing controller 6 generates a vertical timing signal, which determines the timing of vertical scanning of the CCD 1, and a horizontal timing signal, which determines the timing of horizontal scanning. The timing controller 6 controls the operations of the analog processor 2, the A/D converter 3 and the digital processor 4 by a timing clock CT. The timing clock CT is a signal for synchronizing the operations of the individual circuits 2 to 4 with the output operation of the CCD 1.
A driver 5 supplies a multi-phase drive clock to each shift register of the CCD 1 in accordance with the vertical and horizontal timing signals from the timing controller 6. For example, a 4-phase vertical transfer clock .psi.v is supplied to the vertical shift registers, and a 2-phase horizontal transfer clock .psi.h is supplied to the horizontal shift register.
In carrying out color image sensing, attaching a color filter for color separation to the light-receiving surface allows the individual light-receiving pixels of the CCD 1 to be associated with predetermined color components. There are a mosaic type color filter and a stripe type color filter. While the mosaic type color filter has a more complicated structure than the stripe type, it is advantageous over the stripe type in having a higher horizontal resolution. In this respect, it is preferable to use the mosaic type color filters in an image sensing apparatus that needs a high resolution, like a video camera.
FIG. 2 is a schematic plan view of a part of a mosaic type color filter 200. The color filter 200 has a plurality of segments 202 corresponding to the individual pixels of the light-receiving section of the CCD 1. For example, Ye (Yellow), Cy (Cyan), W (White) and G (Green) are cyclically assigned to the individual segments 202. In the example of FIG. 2, the W components and G components are alternately arranged in odd rows, while the Ye components and Cy components are alternately arranged in even rows. In the CCD image sensor 1 equipped with the color filter 200, an image signal having the alternate W and G components is output when odd rows of pixel information are read out, and an image signal having the alternate Ye and Cy components is output when even rows of pixel information are read out.
FIG. 3 is a schematic block diagram of the digital processor 4 used as an image signal processing apparatus. It is assumed in this case that the CCD image sensor 1 is equipped with the color filter 200 shown in FIG. 2.
A color separator 11 receives the image data D output from the A/D converter 3 in accordance with the arrangement order of the color components of the color filter 200, and separates the image data D color component by color component to generate color component data C[Ye], C[Cy], C[G] and C[W]. As shown in FIG. 4, the image data D having the alternate W components and G components is output in an operation of reading odd rows of pixel information, and the image data D having the alternate Ye components and Cy components is output in an operation of reading even rows of pixel information. The color separator 11 outputs all of the color component data C[Ye], C[Cy], C[G] and C[W] at the time of each row of pixel information is read by retaining at least one row of image data D. Specifically, during reading of an odd row, the color separator 11 separates the odd row of image data D into the color component data C[G] and C[W] and at the same time separates the previously read even row of image data D into the color component data C[Ye] and C[Cy]. This scheme causes the color component data to be output intermittently at the time of outputting the image data serially. The intermittent portions of the color component data are however interpolated by outputting the same color component data twice in succession.
A color calculator 12 performs a color computation according to, for example, the following equations on the color component data C[Ye], C[Cy], C[G] and C[W] from the color separator 11, to generate primary color data P[R], P[B] and P[G] corresponding to the three primary colors (R: red, B: blue and G: green) of light. EQU Ye-G=R EQU Cy-G=B EQU G=G
A color balance controller 13 receives the primary color data P[R], P[B] and P[G] from the color calculator 12 and adjusts the color balance of the individual primary color data using predetermined gains specific to the respective primary colors. This adjustment compensates for differences among the sensitivities of the light-receiving pixels of the CCD image sensor 1 which differ color component by color component data, resulting in an improved color reproduction of the reproduced image.
A color difference calculator 14 produces color difference data U and V from the primary color data P[R], P[B] and P[G] from the color balance controller 13. The color difference calculator 14 combines the primary color data P[R], P[B] and P[G] by a ratio of 3:6:1 to generate luminance data. Then, the color difference calculator 14 subtracts the luminance data from the primary color data P[B] corresponding to the B component to generate the color difference data U. Further, the color difference calculator 14 subtracts the luminance data from the primary color data P[R] corresponding to the R component to generate the color difference data V.
A luminance calculator 15 combines the four color components Ye, Cy, G and W included in the image data D from the color separator 11 to generate luminance data Y. That is, EQU Y=Ye+Cy+G+W=(B+G)+(R+G)+G+(R+G+B) =2R+4G+2B
is produced. The luminance data Y results from combination of the R, G and B components by a ratio of 1:2:1. While a luminance signal is generated by combining the R, G and B components by a ratio of 3:6:1 according to the NTSC standards, a ratio close to this ratio does not raise a practical problem.
An aperture circuit 16 enhances a specific frequency component included in the luminance data Y to generate aperture data, and adds this aperture data to the luminance data Y to enhance the outline of the image of a target object. In other words, the aperture circuit 16 performs a filtering process on the image data D to produce aperture data in which the frequency component of 1/4 of the sampling frequency (which is used to acquire the image data D from the image signal Y) is enhanced. The luminance data Y modified by the aperture circuit 16 is supplied together with the color difference data U and V to an external display device or recording device (not shown).
As the process of generating the R and B components and the process of generating the G component in the color computation of the color calculator 12 differ from each other, the color components may vary due to the influence of noise, resulting in degradation of the quality of the reproduced image. When noise is contained in all the Ye, Cy, G and W color components, for example, the noise components are canceled out by the subtracting process at the time of producing the R and B components. At the time of producing the G component, however, the noise components remain. Therefore, the G component is most likely to be affected by noise. When the image of an object is sensed under a low brightness, the influence of noise is enhanced and prominently appears in the reproduced image.
To suppress the influence of noise, it has been proposed to change the color computing scheme of the color calculator 12 as follows. The color component data C[Ye] is subtracted from the color component data C[W] to produce the primary color data P[R]. The color component data C[Cy] is subtracted from the color component data C[W] to produce the primary color data P[B]. Further, the color component data C[Ye] and the color component data C[Cy] are added together, and then the color component data C[W] is subtracted from the added data to produce the primary color data P[G]. According to this scheme, since the color component data C[W] has a higher level than the other color components based on the characteristics of the color filter, noise components contained in the primary color data do not stand out.
Since the W component has a high light-receiving sensitivity, it will reach a saturation level more easily than the other color components. At the time of image-sensing an object under a high brightness, therefore, the R and B components become insufficient. That is, the R and B components generated using the saturated W component have lower levels than the actual R and B components. Further, the G component is acquired by subtracting the W component from the result of adding the Ye and Cy components. If the W component is saturated, then, the G component has a greater level than the actual G component. Consequently, the G component alone is enhanced, leaving the reproduced image with a poor color balance. This narrows the dynamic range of the image sensing device, thereby restricting the optical system.
It is an objective of the present invention to provide an image signal processing apparatus that suppresses the influence of noise in low brightness and maintains a good color balance in high brightness.